Electronic device, storage medium, program, and displaying method

ABSTRACT

An electronic device is provided which displays an object (body) on a flexible display screen in accordance with a three-dimensional shape of the display screen by utilizing the flexibility of the display screen. An electronic device including a display portion which includes a flexible display device displaying an object on a display screen; a detection portion detecting positional data of a given part of the display screen; and an arithmetic portion calculating a three-dimensional shape of the display screen on the basis of the positional data and computing motion of the object to make the object move according to a given law in accordance with the calculated three-dimensional shape of the display screen.

TECHNICAL FIELD

The present invention relates to an electronic device whose displayscreen has flexibility, a storage medium which stores a program forperforming a display in the electronic device, a program for performinga display in the electronic device, and a method for performing adisplay in the electronic device.

BACKGROUND ART

In recent years, portable information terminals such as mobile phones orsmartphones, portable music reproducing devices, portable game machines,and other consumer portable electronic devices with high performancewhich include display devices have become so widespread as to change thelifestyle of the people of today.

Development of such portable electronic devices is further promoted; aswell as multifunctionality of the devices, development in terms ofexternal features, such as a reduction in size or weight, is alsoactively pursued. In addition, as next-generation portable electronicdevices, flexible electronic devices in the form of a sheet are beingactively researched and developed.

The e-book reader disclosed in Patent Document 1 has a flexible housingand a positional sensor and a bending sensor which detect a bended partof the e-book reader. Contents displayed on a display portion areswitched on the basis of results of the detection by the positionalsensor and the bending sensor, whereby a user can control the displaywithout using a mouse or a button owing to the flexibility of thehousing.

REFERENCE Patent Document

[Patent Document 1] Japanese Published Patent Application No.2010-157060

DISCLOSURE OF INVENTION

Meanwhile, in such a flexible portable electronic device, even thoughoperation utilizing the flexibility is possible, displayed still imagesor moving images are flat, where the flexibility of the display screenand the electronic device which includes the display screen is notsufficiently utilized. In this manner, the product planning has failedto achieve a synergistic effect of the flexibility of the display screenand the multifunctionality of the electronic device, which makes theelectronic device including a flexible display portion less attractiveto users.

In view of the above, an object of one embodiment of the presentinvention is to provide an electronic device which displays an object (abody) on a flexible display screen in accordance with athree-dimensional shape of the display screen by utilizing theflexibility of the display screen.

Another object of one embodiment of the present invention is to providea program for displaying an object on a flexible display screen inaccordance with a three-dimensional shape of the display screen byutilizing the flexibility of the display screen.

Thus, one embodiment of structure of the invention disclosed in thisspecification is an electronic device which includes a display portionincluding a flexible display device displaying an object on a displayscreen; a detection portion detecting positional data of a given part ofthe display screen; and an arithmetic portion calculating athree-dimensional shape of the display screen on the basis of thepositional data and computing motion of the object to make the object bedisplayed to move according to a given law in accordance with thecalculated three-dimensional shape of the display screen.

Another embodiment of a structure of the invention disclosed in thisspecification is a computer-readable storage medium storing a programfor making an electronic device which includes a flexible display devicedisplaying an object on a display screen execute a first step ofdetecting positional data of a given pail of the display screen; asecond step of calculating a three-dimensional shape of the displayscreen on the basis of the positional data; a third step of computingmotion of the object to make the object be displayed to move accordingto a given law in accordance with the calculated three-dimensional shapeof the display screen; and a fourth step of displaying the computedmotion of the object on the display screen.

A further embodiment of a structure of the invention disclosed in thisspecification is a program for making an electronic device whichincludes a flexible display device displaying an object on a displayscreen execute a first step of detecting positional data of a given partof the display screen; a second step of calculating a three-dimensionalshape of the display screen on the basis of the positional data; a thirdstep of computing motion of the object to make the object be displayedto move according to a given law in accordance with the calculatedthree-dimensional shape of the display screen; and a fourth step ofdisplaying the computed motion of the object on the display screen.

A still further embodiment of a structure of the invention disclosed inthis specification is a displaying method which uses an electronicdevice including a flexible display device displaying an object on adisplay screen and includes a step of detecting positional data of agiven part of the display screen; a step of calculating athree-dimensional shape of the display screen on the basis of thepositional data; a step of computing motion of the object to make theobject be displayed to move according to a given law in accordance withthe calculated three-dimensional shape of the display screen; and a stepof displaying the computed motion of the object on the display screen.

An electronic device according to one embodiment of the presentinvention includes, in its category, a portable electronic device suchas a mobile phone, a PHS, a smartphone, a personal computer, a personaldigital assistant (PDA), a tablet PC, a laptop PC, a mini computer, ane-book reader (electronic paper), an electronic dictionary, anelectronic notebook, a calculator, a navigation system, a digital photoframe, an image reproducing device, an IC recorder, or a portable gamemachine.

In a flexible display screen and an electronic device which includes theflexible display screen, a detection portion in which a plurality ofsensors are provided in matrix, for example, is provided directly undera display device including the flexible display screen. Positional datadetected by the sensors and the like are combined and a curved shape ofa display portion is calculated. An object is displayed to moveaccording to a given law in accordance with the curved shape of thedisplay screen (a three-dimensional shape of the display screen).

Note that a “three-dimensional shape of the display screen” refers to aphysical form of the display screen which is altered (e.g., curved orbended) by application of external force on the electronic device or thedisplay device, which can be defined by three-dimensional spacecoordinates. Thus, the three-dimensional shape also includes, in itscategory, a two-dimensional shape before or after deformation (i.e., aflat shape without a curve).

“An object is displayed to move according to a given law in accordancewith a three-dimensional shape of the display screen” means that, forexample, in the case where the display screen is warped downward, theobject (body) displayed on the display screen is displayed to move inaccordance with force imaginarily applied to the object, which is, forexample, force in the natural world such as gravity. The object may be asolid (e.g., a dice, a fallen leaf, or a marble) or liquid (e.g.,water). Further, all things in nature including gas and powder can bedefined as the object. Accordingly, a user can feel a sense of realityas if the object exists on the surface of the electronic device.

Moreover, the object may be a living thing. For example, in electronicgames or the like, the object can be a character in the form of aperson, an animal, a plant, or an imaginary creature.

A “given law” refers to a law which can be expressed by a givenequation. For example, it refers to a physical law of the natural world,specifically a law which defines motion of a body. More specificexamples are laws, which can be expressed by equations of motionvisually descriptive of mechanics, of gravity, attractive force,frictional force, air resistance, motion of a rigid or fluid body, andthe like. However, a “given law” is not limited to physical laws of thenatural world. For example, to enhance a visual effect, the given lawmay be a law which is accentuated with deviation from a physical law ofthe natural world, or a law violating the natural law, by which anobject floats defying gravity, for example. In addition, such, anequation does not necessarily represent a physical law of the naturalworld exactly, and may be a pseudo equation or an equation simplified.

The detection portion has sensors which are provided in a plurality ofgiven positions for calculation of a three dimensional shape of thedisplay screen of the electronic device. For example, it is possible toprovide a plurality of positional sensors in matrix in the vicinity ofthe display screen as the sensors so that the positional sensors detectrelative positional coordinates. It is also possible to provide aplurality of acceleration sensors in matrix in the vicinity of thedisplay screen as the sensors so that the acceleration sensors detectrelative change in acceleration of each part, which accompaniesdeformation of the display screen. The sensors are not limited to theabove ones and can be sensors to which, for example, a mechanical,electromagnetic, thermal, acoustic, or chemical means is applied as longas the sensors can detect parameters necessary for the calculation of athree-dimensional shape of the display screen. For example, anacceleration sensor, an angular velocity sensor, a vibration sensor, apressure sensor, a gyroscope sensor, or the like can be used as thesensors. Alternatively, these sensors may be combined to be used.

Note that the present invention includes, in its scope, a method,hardware (e.g., an electronic device, a computer, a semiconductordevice, a storage medium), a system, a program, software, and the likein which the operations in the present invention are achieved.

An electronic device can be provided which displays an object on aflexible display screen in accordance with a three-dimensional shape ofthe display screen by utilizing the flexibility of the display screen.

A program for displaying an object on a flexible display screen inaccordance with a three-dimensional shape of the display screen byutilizing the flexibility of the display screen can be provided.

Thus, a user interface (UI) can be provided which can offer a user asense of reality as if an object displayed on the display screen existson the surface of the electronic device.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A to 1D illustrate one mode of an electronic device.

FIG. 2 is a block diagram illustrating one mode of a hardware structureof an electronic device.

FIG. 3 is a block diagram illustrating a structure of a memory.

FIG. 4 is a functional block diagram illustrating one mode of anelectronic device.

FIGS. 5A and 5B illustrate data structures.

FIGS. 6A to 6C illustrate motion of an object displayed on a displayscreen.

FIG. 7 is a flow chart illustrating steps of displaying an object.

FIG. 8 is a flow chart illustrating steps of displaying an object.

FIG. 9 is a flow chart illustrating steps of displaying an object.

FIG. 10 is a flow chart illustrating steps of displaying an object.

FIG. 11 is a flow chart illustrating steps of displaying an object.

FIG. 12 is a flow chart illustrating steps of displaying an object.

FIG, 13 is a flow chart illustrating steps of displaying an object.

FIGS. 14A to 14C illustrate motion of an object displayed on a displayscreen.

FIGS. 15A and 15B illustrate motion of an object displayed on a displayscreen.

FIGS. 16A and 16B each illustrate motion of an object displayed on adisplay screen.

FIGS. 17A and 17B each illustrate motion of an object displayed on adisplay screen.

FIGS. 18A and 18B each illustrate motion of an object displayed on adisplay screen.

FIG. 19 is a flow chart illustrating steps of displaying an object.

FIGS. 20A and 20B illustrate motion of objects displayed on a displayscreen.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the invention disclosed in thisspecification will be described with reference to the accompanyingdrawings. Note that the invention disclosed in this specification is notlimited to the following description, and it is easily understood bythose skilled in the art that modes and details can be variously changedwithout departing from the spirit and the scope of the invention.Therefore, the invention disclosed in this specification is notconstrued as being limited to the description of the followingembodiments.

Embodiment 1

In this embodiment, one mode of a structure of an electronic devicewhich has a flexible display screen and one mode of a method forperforming a display on the display screen are described with referenceto FIGS. 1A to 1D, FIG. 2, FIG. 3, FIG. 4, FIGS. 5A and 5B, FIGS. 6A to6C, FIG. 7, FIG. 8, FIG. 9, FIG. 10, FIG. 11, FIG. 12, and FIG. 13.

(Structure of the Electronic Device)

An example of a structure of the electronic device which has a flexibledisplay screen according to one embodiment of the present invention isdescribed with reference to FIGS. 1A to 1D. This embodiment describes anexample in which the electronic device is a portable informationterminal capable of executing a variety of applications such as mobilephone calls, e-mailing, viewing and editing texts, music reproduction,Internet communication, and a computer game. FIG 1A is a top view of anelectronic device 100. The electronic device 100 includes a housing 101,a display screen 102, and a home button 103.

The display screen 102 is a part of a display device which displaysimages including a still image and a moving image. As the display deviceincluding the display screen 102, there are a light-emitting device inwhich each pixel includes a light-emitting element typified by anorganic light-emitting element (OLED); a liquid crystal display device;an electronic paper performing a display in an electrophoretic mode, anelectronic liquid powder (registered trademark) mode, or the like; adigital micromirror device (DMD); a plasma display panel (PDP); a fieldemission display (FED); a surface conduction electron-emitter display(SED); a light-emitting diode (LED) display; a carbon nanotube display;a nanocrystal display; a quantum dot display; and the like. The displayscreen according to one embodiment of the present invention is a part ofany of these display devices, and a flexible display device is used asthe display device including the display screen.

In this embodiment, a touch panel with which data can be input by aninstruction means such as a finger or a pen is provided as an inputmeans on the display screen 102. Since the touch panel is provided, aregion for a keyboard on the electronic device becomes unnecessary andthus the display screen can be provided in a large region. Moreover,since data can be input with a pen or a finger, a user-friendlyinterface can be obtained. Although the touch panel may be of any ofvarious types such as a resistive type, a capacitive type, an infraredray type, an electromagnetic induction type, and a surface acoustic wavetype, a resistive type or a capacitive type is particularly preferablesince the display screen 102 according to one embodiment of the presentinvention can be curved.

Since the display screen 102 is flexible, the housing 101 also needs tohave deformability. The housing 101 is preferably formed using anelastic resin material, a plastically deformable metal material, acombination thereof, or the like. For example, a pressed metal plate canbe used for the four corners of the housing 101 and a plastic moldedbody can be used for the other parts. Note that although not shown, itis also possible to use a material which is not flexible for the housing101 when only the display device including the display screen 102 hasflexibility and a space is provided between the display device and thehousing 101. In this case, the housing 101 may be partly pleated, forexample, so that the housing 101 can be stretched and shrunk as thedisplay screen 102 is curved.

As illustrated in FIG. 1A, the electronic device 100 described in thisembodiment has a rectangular shape in which a long side is longer than ashort side. The reason for this is that with this shape, curving in adirection perpendicular to the long side is particularly easilyperformed, so that a user can feel a feature of the flexibility of theelectronic device 100 more strongly. However, by a 90° turn of theelectronic device 100, it can be used as a vertically oriented displaydevice with a short side at the bottom. To realize this, an accelerationsensor or the like may be provided in the electronic device 100, inwhich case the acceleration sensor detects rotation of the electronicdevice 100 and a display on the display screen 102 is switched from alandscape orientation to a portrait orientation.

The four corners of the electronic device 100 are rounded. Since theelectronic device 100 has its four corners rounded, concentration ofstress at edges of the four corners due to bending or twisting can berelieved, which can lead to an improvement in durability of the displaydevice and the electronic device 100.

Further, so that the electronic device 100 can be easily curved, thethickness of the electronic device 100 is preferably as small aspossible in a range which ensures a certain strength.

The shape of the electronic device 100 which is illustrated in FIG. 1Ais merely an example and the present invention is not limited thereto; asquare, a circle, an ellipse, or the like can be employed as demanded byusers.

The home button 103 is provided on the lower middle part of the uppersurface of the housing of the electronic device 100. When the homebutton 103 is pressed, a home screen is displayed on the display screen102. Further, the electronic device 100 may be configured such that mainpower supply of the electronic device 100 is turned off with a press ofthe home button 103 for a given time. A structure may also be employedin which a press of the home button 103 brings the device which is in asleep mode out of the sleep mode. Besides, the home button can be usedas a switch for starting a variety of functions, for example, inaccordance with the length of time for pressing or by pressing the homebutton at the same time as another button. When the home button 103 hasa variety of functions as described above, the buttons actually providedon the housing 101 can be reduced in number so as to simplify thestructure and the design of the electronic device 100. The number ofbuttons provided on the housing 101 is preferably made as small aspossible so that the electronic device 100 according to one embodimentof the present invention, which has flexibility, becomes lesstrouble-prone and has high reliability.

The undersurface (bottom surface) of the electronic device 100 isillustrated in FIG. 1B. The undersurface of the electronic device 100has a slightly smaller area than the upper surface. That is, a sidesurface of the electronic device 100 has a shape which slopes from theupper surface to the undersurface. As illustrated in FIG. 1B, the sidesurface of the electronic device 100 has a plurality of input-outputfunctions including buttons.

A volume control button 104 and a mute button 105 are provided on theupper right corner of the side surface of the electronic device 100. Aspeaker 107 for outputting sound is provided on the lower left corner ofthe side surface of the electronic device 100. The speaker 107 outputsvarious kinds of sound, examples of which are sound set forpredetermined processing such as startup sound of an operating system(OS), sound from sound files executed in various applications, such asmusic from music reproduction application software, and an incominge-mail alert. Specifically, in the electronic device 100 according toone embodiment of the present invention, the speaker 107 may outputsound in response to a curve of the display screen 102, or may outputsound in response to move of an object, which will be described later.

In addition, the volume of sound output from the speaker 107 can beadjusted with the volume control button 104. Sound from the speaker 107can be silenced instantaneously by a press of the mute button 105.Although not shown, a connector for outputting sound to a device such asheadphones, earphones, or a headset may be provided together with orinstead of the speaker 107 for outputting sound.

A microphone 106, which can be used for sound input and recording, isprovided on the lower right corner of the side surface of the electronicdevice 100. Further, a sleep button 108 is provided on the upper leftcorner of the side surface of the electronic device 100. The electronicdevice 100 can be put into the sleep mode when the sleep button 108 ispressed, in which case power can be saved by stopping given functionssuch as a display on the display screen 102 with the main power supplyon. In this manner, consumption of power of a battery incorporated inthe electronic device 100 can be suppressed.

When a variety of input-output devices are locally provided on the fourcorners of the electronic device 100 as described above, parts which donot have flexibility can be concentrated in the four corners, whichallows the electronic device 100 to have flexibility as a whole. In thecase where a non-flexible member is used for the four corners, thestructural strength of the electronic device 100 increases and theusability of the electronic device 100 can be enhanced. Therefore, forthe four corners of the housing 101 of the electronic device 100, anon-flexible member which is different from materials of the other partsis preferably used.

In contrast, a camera 109 is provided on the upper middle part of theundersurface of the electronic device 100. Images obtained with the useof the camera 109 can be displayed on the display screen 102. Note thatproviding the camera 109 on the middle part allows the user to knowwhere the camera 109 is positioned even in the case where the useroperates the electronic device 100 while seeing its upper surface. Sincethe electronic device 100 according to one embodiment of the presentinvention has flexibility, it is possible to capture blur-free images bystanding the electronic device 100 which is curved into a U shape on atable or the like, without using a fixture such as a tripod.

Note that although not shown, the electronic device 100 may include aterminal for connecting an external memory drive. Examples of theexternal memory drive are storage medium drives such as an external harddisk drive (HDD), a flash memory drive, a digital versatile disk (DVD)drive, a DVD-recordable (DVD-R) drive, a DVD-rewritable (DVD-RW) drive,a compact disc (CD) drive, a compact disc recordable (CD-R) drive, acompact disc rewritable (CD-RW) drive, a magneto-optical (MO) discdrive, a floppy disk drive (FDD), and a nonvolatile solid state drive(SSD) device different from the above flash memory drive. Although theelectronic device 100 has the touch panel on the display screen 102, akeyboard may be provided on the housing 101 instead of the touch panelor may be externally added.

The above-described electronic device 100 has flexibility as illustratedin FIGS. 1C and 1D. For example, FIG. 1D illustrates the electronicdevice 100 whose right and left end portions are curved upward. In thismanner, the housing 101 and the display screen 102 can be curved. Notethat the flexibility of the electronic device in this specificationachieves at least part of the effect of the invention disclosed in thisspecification, and the degree, direction, or the like of curving of theelectronic device is not particularly limited.

FIG. 2 is an example of a block diagram illustrating a hardwarestructure of the electronic device 100 in this embodiment which hasflexibility. The electronic device 100 includes a processor 151, a mainmemory 152, a memory controller 153, an auxiliary memory 154, a sensorcontroller 155, sensors 156, a display controller 157, a display device158, a power supply controller 159, a power supply 160, a communicationcontroller 161, a communication interface (I/F) 162, a sound controller163, a speaker 164, a sound output connector 165, a microphone 166, aninput interface 167, a housing switch 168, a touch panel 169, a keyboard170, a camera 171, an external port 172, an output interface 173, and avibration motor 174. Among these, the processor 151, the main memory152, the memory controller 153, the sensor controller 155, the displaycontroller 157, the power supply controller 159, the communicationcontroller 161, the sound controller 163, the input interface 167, andthe output interface 173 are connected to one another through one ormore system buses 150 and can communicate with one another.

The above structure of the electronic device 100 is merely an exampleand some of the components can be omitted; for example, the keyboard 170can be omitted, in which case a virtual keyboard using the touch panel169 and the display device 158 is produced with software. Moreover, acomponent other than the above may be added to the structure.

For the processor 151, a microprocessor such as a digital signalprocessor (DSP) or a graphics processing unit (GPU) can be used inaddition to a central processing unit (CPU). The processor 151interprets and executes instructions from various programs to processvarious kinds of data and control programs.

Note that a thin film transistor in which a channel formation regionincludes an oxide semiconductor can be used for the processor 151.Because the transistor has extremely low off-state current, by using thetransistor as a switch for holding electric charge (data) flowed into amemory element, a long data retaining period can be ensured. Byutilizing the above characteristics for a register of the processor 151or the like, the processor 151 can operate only when needed and thecontents of the previous processing can be stored in the memory elementin the rest of the time, so that normally off computing can beperformed; thus, power consumption of the electronic device can bereduced.

The main memory 152 is used as a main memory device. FIG. 3 is a blockdiagram illustrating a structure of the main memory 152. The main memory152 has a volatile memory and a nonvolatile memory such as a accessmemory (RAM) 180 and a read only memory (ROM) 181.

A dynamic random access memory (DRAM), for example, is used for the RAM180, and a memory space as a workspace for the processor 151 isvirtually allocated and used. As illustrated in FIG. 3, an operatingsystem 182, an application program 183, a program module 184, programdata 185, and the like which are stored in the auxiliary memory 154,which is an HDD or the like, are loaded into the RAM 180 to be executed.The data, program, and program module which are loaded into the RAM 180are directly accessed and operated by the processor 151. Note that inFIG. 3, a memory controller for controlling the data or the like storedin the RAM 180 is assumed to be incorporated in the processor 151 and isnot illustrated; however, a memory controller for controlling the mainmemory 152 may be separately provided.

In the ROM 181, a basic input/output system (BIOS) 186, firmware, andthe like for which rewriting is not needed are stored. Further, displayportion physical property data 187 including physical parameters of thedisplay screen 102 having flexibility according to one embodiment of thepresent invention, and sensor characteristics data 188 concerning thecharacteristics of the sensors 156 which detect positional data can bestored in the ROM 181 in advance. As the ROM 181, a mask ROM, a one timeprogrammable read only memory (OTPROM), or an erasable programmable readonly memory (EPROM) can be used. As an EPROM, an ultra-violet erasableprogrammable read only memory (UV-EPROM) which can erase stored data byirradiation with ultraviolet rays, an electrically erasable programmableread only memory (EEPROM), a flash memory, and the like can be given.

The auxiliary memory 154 incorporated in the electronic device 100functions as an auxiliary memory device. The auxiliary memory 154 is astorage medium having a larger capacity than the main memory 152, and isconnected to the system bus 150 through the memory controller 153. Thememory controller 153 functions as an interface which controls readingand writing of data from and into the auxiliary memory 154, for example.For the auxiliary memory 154, a storage medium drive such as a hard diskdrive (HDD) or a nonvolatile solid state drive (SSD) device can be used,for example.

Note that although being incorporated in the electronic device 100 inFIG. 2, the auxiliary memory 154 may be an external memory device whichis provided outside the electronic device 100 and connected through theexternal port 172; moreover, the external memory device and theauxiliary memory 154 may be combined to be used as the auxiliary memorydevice.

The sensors 156 detect parameters necessary for calculating athree-dimensional shape of the display screen of the electronic device100. For example, as the sensors 156, a plurality of positional sensorswhich can specify a positional relation relative to each other can beprovided in matrix in the vicinity of the display screen so that thepositional sensor detects positional data relative to the otherpositional sensors. It is also possible to provide a plurality ofacceleration sensors in matrix in the vicinity of the display screen asthe sensors 156 so that the acceleration sensors detect relative changein acceleration of each part, which accompanies deformation of thedisplay screen. The sensors 156 are not limited to the above ones andcan be sensors to which, for example, a mechanical, electromagnetic,thermal, acoustic, or chemical means is applied as long as the sensorscan detect parameters necessary for the calculation of athree-dimensional shape of the display screen. For example, anacceleration sensor, an angular velocity sensor, a vibration senor, apressure sensor, a gyroscope sensor, or the like can be used as thesensors. Alternatively, these sensors may be combined to be used. Notethat the sensors 156 may be incorporated in touch sensors provided onthe display screen 102. By combining the touch sensors and thepositional sensors into one component, the number of parts can bereduced, which can contribute to a reduction in the thickness of theelectronic device 100.

The sensor controller 155 is an interface which performs centralizedcontrol of the plurality of sensors 156. The sensor controller 155supplies power from the power supply 160 to the plurality of sensors156, and receives input from the sensors 156, converts it into a controlsignal, and outputs the signal to the system bus 150. The sensorcontroller 155 may handle errors made by the sensors 156 or maycalibrate the sensors 156.

The display device 158 is connected to the system bus 150 through thedisplay controller 157. As the display device 158, a display devicehaving flexibility and selected from a light-emitting device in whicheach pixel includes a light-emitting element typified by an organiclight-emitting element (OLED); a liquid crystal display device; anelectronic paper performing a display in an electrophoretic mode, anelectronic liquid powder (registered trademark) mode, or the like; adigital micromirror device (DMD); a plasma display panel (PDP); a fieldemission display (FED); a surface conduction electron-emitter display(SED); a light-emitting diode (LED) display; a carbon nanotube display;a nanocrystal display; a quantum dot display; and the like is used. Inresponse to drawing instructions input from the processor 151 throughthe system bus 150, the display controller 157 controls the displaydevice 158 so that a predetermined image is displayed on the displayscreen 102 of the display device 158.

The power supply 160 supplies power to a plurality of components of theelectronic device 100. As the power supply 160, for example, one or moreprimary batteries or secondary batteries are included. In the case ofindoor use or the like, an alternate-current (AC) power supply may beused as an external power supply. Particularly in the case of using theelectronic device 100 separately from the external power supply, it isfavorable that the power supply have a large charge/discharge capacitywhich allows the electronic device 100 to be used for a long time. Whenthe power supply 160 is charged, a charger separate from the electronicdevice 100 may be used. Further, since the electronic device 100 in thisembodiment is flexible, it is preferable that the power supply 160 bealso flexible. As a secondary battery having such a feature, forexample, a lithium ion secondary battery and a lithium ion polymersecondary battery can be given. It is preferable that a laminate packagebe used as a jacket of the battery so that the battery has flexibility.

Further, although not shown, the power supply 160 may have a powersupply management device (battery management unit: BMU). The BMUcollects data on cell voltage or cell temperatures of the battery,monitors overcharge and overdischarge, controls a cell balancer, handlesa deterioration state of the battery, calculates the remaining batterypower (state of charge: SOC), and controls detection of a failure, forexample.

The power supply controller 159 controls transmission of power from thepower supply 160 to each component through the system bus 150 or a powersupply line. The power supply controller 159 has a power converter or aninverter with a plurality of channels, a protection circuit, and thelike. Further, the power supply controller 159 has a function ofreducing power consumption. For example, after detection of no input tothe electronic device 100 for a given period, the power supplycontroller 159 lowers clock frequency or stops input of clocks of theprocessor 151, stops operation of the processor 151 itself, or stopsrotation of the HDD, thereby reducing the power consumption. Such afunction is performed with the power supply controller 159 alone or thepower supply controller 159 interlocking with the processor 151.

The communication interface (I/F) 162 is connected to the system bus 150through the communication controller 161. The communication controller161 and the communication I/F 162 control, in response to instructionsfrom the processor 151, a connection signal for connecting theelectronic device 100 to a computer network, and transmit the signal tothe computer network. Accordingly, communication can be performed byconnecting the electronic device 100 to a computer network such as theInternet (which is an infrastructure of the World Wide Web (WWW)), anintranet, an extranet, a personal area network (PAN), a local areanetwork (LAN), a campus area network (CAN), a metropolitan area network(MAN), a wide area network (WAN), or a global area network (GAN).

In the case where communication between the electronic device 100 andanother device is performed without a transmission line, i.e.,wirelessly, a radio-frequency (RF) circuit may be provided in thecommunication interface (I/F) 162 so that an RF signal is transmittedand received. The RF circuit performs conversion between anelectromagnetic sign and are electric signal in a frequency band whichis set by a national law, and performs communication with anothercommunication device wirelessly with the use of the electromagneticsignal. Several tens of kilohertz to several tens of gigahertz is apractical frequency band which is generally used. The RF circuitincludes an RF circuit portion and an antenna which are compatible witha plurality of frequency bands; the RF circuit portion can include anamplifier, a mixer, a filter, a digital signal processor (DSP), an RFtransceiver, or the like. In the case of performing wirelesscommunication, it is possible to use, as a communication protocol or acommunication technology, a communications standard such as GlobalSystem for Mobile Communication (GSM) (registered trademark), EnhancedData Rates for GSM Evolution (EDGE), Code Division Multiple Access 2000(CDMA2000), or Wideband Code Division Multiple Access (W-CDMA), or acommunications standard developed by IEEE such as Wireless Fidelity(Wi-Fi) (registered trademark), Bluetooth (registered trademark), orZigBee (registered trademark).

Further, in the case where the electronic device 100 is used as a phonefor a telephone conversation, the communication controller 161 and thecommunication I/F 162 control, in response to instructions from theprocessor 151, connection signals for connecting the electronic device100 to a telephone line, and transmit the signal to the telephone line.

The speaker 164, the sound output connector 165, and the microphone 166,which are responsible for sound, are connected to the sound controller163 to be connected to the processor 151 through the system bus 150. Thesound controller 163 generates analog sound signals audible to the userin response to instructions from the processor 151, and outputs thesignals to the speaker 164 or the sound output connector 165. Sound datainput to the microphone 166 is converted into a digital signal in thesound controller 163 and processed in the sound controller 163 and theprocessor 151. To the sound output connector 165, a sound output devicesuch as headphones, earphones, or a headset is connected and soundgenerated in the sound controller 163 is output to the device.

One or more switches provided on the housing (hereinafter refereed to asthe housing switches 168 for convenience), the touch panel 169 providedin the vicinity of the display screen 102, the keyboard 170 provided onthe housing 101, the camera 171 provided on the undersurface of thehousing 101, and the external port 172 to which other input componentscan be connected are controlled by the input interface 167, and theinput interface 167 is connected to the processor 151 and the likethrough the system bus 150.

The housing switches 168 correspond to, for example, the home button103, the volume control button 104, the mute button 105, the sleepbutton 108, and the like which are illustrated in FIGS. 1A to 1D. Aswell as these housing switches 168, the touch panel 169, the keyboard170, the camera 171, and the external port 172, the microphone 166 forsound input and the sensors 156 detecting change in the shape of thedisplay screen 102 serve as interfaces between the user and theelectronic device 100.

The touch panel 169 is provided on the display screen 102 and can beused as an input device by which data is input with an instruction meanssuch as a finger or a pen. Since the touch panel 169 is provided, aregion for a keyboard on the electronic device becomes unnecessary andthus the display screen can be provided in a large region. Moreover,since data can be input with a pen or a finger, a user-friendlyinterface can be obtained. Although the touch panel 169 may be of any ofvarious types such as a resistive type, a capacitive type, an infraredray type, an electromagnetic induction type, and a surface acoustic wavetype, a resistive type or a capacitive type is particularly preferablesince the display screen 102 according to one embodiment of the presentinvention can be curved. Note that the touch panel 169 may include thesensors 156 to serve as one component. This structure allows the numberof parts to be reduced and contributes to a reduction of the thicknessof the electronic device 100.

The vibration motor 174 is connected to the system bus 150 through theoutput interface 173. In response to instructions from the processor151, the output interface 173 controls vibration time and the like andmakes the vibration motor 174 vibrate. Accordingly, the electronicdevice 100 is made to vibrate and the vibration can be used as hapticeffects to the user at the time of reception of e-mails or execution ofapplications such as computer games. The vibration motor 174 may be madeto vibrate in accordance with the degree of a curve of the displayscreen 102; for example, when the flexibility of the display screen 102is limited to a certain degree, the vibration motor 174 can be used toalert the user that the threshold value is exceeded. Although not shown,as well as the vibration motor 174, a variety of output devices withwhich the user can perceive by the use of the five senses can beconnected to the output interface 173. For example, a light-emittingdevice for showing an operation status of the electronic device 100, anaroma diffuser which releases fragrance in response to vibration, or thelike can be connected to the output interface 173.

(Functions of the Electronic Device)

Next, FIG. 4 shows a block diagram illustrating main functions of theelectronic device 100 in this embodiment. The electronic device 100 hasat least four function blocks of a display portion 201, a detectionportion 202, an arithmetic portion 203, and a storage portion 204. Aninput portion 205 and an output portion 206 may be additionallyincluded.

The display portion 201 includes the display device 158 and the displaycontroller 157 described with reference to FIG. 2, and the like, anddisplays a field, an object, or the like on the display screen of thedisplay device 158. At least the display device 158 has flexibility andcan be deformed. On the display screen 102 of the display device 158, anobject which moves in accordance with change in the shape of the displayscreen 102 is displayed. The object here is a body displayed on thedisplay screen 102 and being in motion in response to the deformation ofthe display screen 102. Further, the field is a background of the objectin motion and is a ground that affects the motion of the object. Agraphic expression of the field may be an arrangement of textures thatrepresent materials or may be transparent.

The detection portion 202 includes the sensors 156 and the sensorcontroller 155 described with reference to FIG. 2, and the like, anddetects positional data of the display screen 102 of the display device158. For example, the plurality of sensors 156 are provided in matrixand, at the respective provision sites, each of the sensors 156 obtainspositional data relative to the other sensors 156. The positional dataobtained by the sensors 156 is output to the arithmetic portion 203through the sensor controller 155.

The arithmetic portion 203 includes the processor 151 described withreference to FIG. 2, and the like. To the arithmetic portion 203, thepositional data of the sites of the display screen 102 which is outputfrom the detection portion 202 is input, and the arithmetic portion 203calculates a three-dimensional shape of the display screen 102 on thebasis of the data. The calculation of a three-dimensional shape of thedisplay screen 102 may be performed as appropriate in consideration ofthe sensors 156 or a mode of the calculation to be used. At the time ofthe calculation of a three-dimensional shape of the display screen 102,when comparison with the latest data of data on a three-dimensionalshape of the display screen 102 which has already been calculated isperformed and the amount of change is computed to obtain a newthree-dimensional shape of the display screen 102, computational burdenson the processor 151 can be reduced. At least the latest data of thedata on the three-dimensional shape of the display screen 102 which hasalready been calculated is stored in the storage portion 204. It is alsopossible to calculate a three-dimensional shape of the display screen102 by comparison between positional data which has already beenobtained by the sensors 156 and positional data newly obtained by thesensors 156 and addition of the amount of change to thethree-dimensional shape of the display screen 102. In that case, atleast the latest data of the positional data which has already beenobtained by the sensors 156 is stored in the storage portion 204.

After calculating the three-dimensional shape of the display screen 102,the arithmetic portion 203 makes the object move on the field inaccordance with the calculated three-dimensional shape of the displayscreen 102.

The storage portion 204 includes the main memory 152, the auxiliarymemory 154, and the memory controller 153 described with reference toFIG. 2, and the like. In the storage portion 204, at least data on theobject, data on the field, and data on a law which defines move of theobject are stored. These pieces of data may be included in the operatingsystem, or stored in the storage portion 204 as an application program,a program module, or program data. These pieces of data are stored inthe HDD used as the auxiliary memory 154, for example, and loaded intothe main memory 152 which includes a DRAM and the like as needed bystarting up the electronic device 100.

As illustrated in FIG. 5A, the data on the object (object data 250) isan object ID 251, an object shape 252, an object physical quantity 253such as a set mass or a set surface state (coefficient of friction), anobject image 254, and default 255 of the object such as an initialposition; these pieces of data can be stored in the storage portion 204as a structured list, table, or database. In an example shown in FIG.5A, the data is associated with the object ID 251 to form a layeredstructure.

As illustrated in FIG. 5B, the data on the field (field data 260) is afield ID 261, a field shape 262, a field physical quantity 263, a fieldimage 264, and default 265 of the field; these pieces of data can bestored in the storage portion 204 as a structured list, table, ordatabase. In an example shown in FIG. 5B, the data is associated withthe field ID 261 to form a layered structure.

The law which defines move of an object refers to a law which can beexpressed by a given equation which defines the move of the object. Forexample, it refers to a physical law of the natural world, specificallya law which defines motion of a body. More specific examples are laws,which can be expressed by equations of motion visually descriptive ofmechanics, of gravity, attractive force, frictional force, airresistance, motion of a rigid or fluid body, and the like. However, a“given law” is not limited to physical laws of the natural world. Forexample, to enhance a visual effect, the given law may be a law which isaccentuated with deviation from a physical law of the natural world, ora law violating the natural law, by which an object floats defyinggravity, for example. In addition, such an equation does not necessarilyrepresent a physical law of the natural world exactly, and may be apseudo equation or an equation simplified. Data on a law which definesmove of an object is a group of equations which form the basis forsimulation of the motion of the object. Although not shown, the data onthe law which defines the move of the object can also be stored in thestorage portion 204 as a structured list, table, or database.

The arithmetic portion 203 refers to the physical quantities, positionaldata, and the like which are associated with the object ID and field IDstored in the storage portion 204 and the three-dimensional shape of thedisplay screen 102 which is calculated on the basis of data from thedetection portion 202, and simulates motion of the object on the basisof the data on the law which defines the move of the object so that theobject moves according to the above law. In other words, in response tochange in the shape of the display screen 102, the arithmetic portion203 calls the object data, the field data, the data on the law whichdefines the move of the object, and the like as well as data on thechange in the shape to the workspace, inserts parameters in equations ofthe data on the law which defines the move of the object, and calculatesthe motion of the object. While the calculation of the motion of theobject is performed until the motion of the object substantiallysubsides, the detection portion 202 monitors further change in the shapeof the display screen 102 and modifies the motion of the objectimmediately on detecting change in the shape.

As described above, modification which accompanies the change in theshape of the display screen 102 is repeatedly performed and simulationcontinues until the motion of the object subsides (i.e., the motion ofthe object stops), whereby the realistic motion of the object inaccordance with the deformation of the display screen 102 can be felt bythe user. The motion of the object which is simulated by the arithmeticportion 203 is output to the display portion 201 and displayed on thedisplay screen 102. Note that it is preferable that change in the shapeof the display screen 102 have a given threshold value and thearithmetic portion 203 perform computation only when the threshold valueis exceeded. In that case, an increase in the calculation amount of thearithmetic portion 203 can be suppressed and the motion of the objectcan be stopped at a predetermined level.

The input portion 205 includes the microphone 166, the housing switch168, the touch panel 169, the keyboard 170, the camera 171, the externalport 172, the sound controller 163, and the input interface 167 whichare described with reference to FIG. 2, and the like. For example, theobject displayed on the display screen 102 can move in response to soundinput with the microphone 166. Further, although sensors for calculatingthe shape of the display screen 102 are given as the sensors 156, besidethe sensors, sensors as an input interface can be used. For example, byusing acceleration sensors for the electronic device 100, the object canbe made to move in accordance with the inclination of the electronicdevice 100. The sensors 156 may be used as such sensors for input.

The output portion 206 includes the speaker 164, the sound outputconnector 165, the sound controller 163, the vibration motor 174, andthe output interface 173 which are described with reference to FIG. 2,and the like. For example, while the object whose motion is determinedas described above moves, the electronic device 100 is made to vibrateby control of the vibration motor 174, thereby allowing the user to feela sense of reality through a haptic sense.

(Example of Operation of the Electronic Device)

Next, an example of operation of the electronic device 100 is describedwith reference to FIGS. 6A to 6C.

FIG. 6A is a perspective view illustrating the upper surface of theelectronic device 100, and an object 301 is displayed on the displayscreen 102 of the electronic device 100. In order to describe displayoperation for the object 301, other displays are not shown in thedrawing; however, any other display matters such as a background image,an icon, a toolbar, a pointer, is window, a text, a moving image, or aweb browser can be actually displayed at the same time as the object301.

The object 301 in FIG. 6A is designed to resemble a spherical solid andis at rest at a given position. The range of motion of the object 301 isdefined as a field, which functions as a ground that affects the motionof the object 301.

FIG. 6B illustrates the electronic device 100 whose right end portion iscurved by being lifted. By the deformation of the electronic device 100,the display screen 102 is also deformed. Data from the plurality ofsensors 156 which have detected the deformation is combined and thearithmetic portion 203 calculates the changed shape of the displayscreen 102. Further, various kinds of data is called from the storageportion and the motion of the object is simulated by the above-describedmethod. The simulation results are displayed on the display screen 102as the motion of the object. In FIG. 6B, since the right side of thedisplay screen 102 is lifted, the object 301 moves to the center of thedisplay screen 102 as if it is attracted by gravity (in a directionrepresented by an arrow in the drawing).

In FIG. 6B, the object 301 is a solid sphere. Thus, under Newtonmechanics, the object 301 rolls to the center of the display screen 102.Here, the object 301 has a given physical quantity. Therefore, themoving speed and the rolling velocity depend on the mass or the like.The field also has a predetermined physical quantity or the like. Forexample, the object 301 moves while encountering air resistance set forthe field, gravity acceleration set for the field, and friction set forthe field.

Applying various laws to the motion of the object 301 as described aboveallows the user to feel a sense of reality.

Note that all of physical quantities and the like of the object 301 andthe field are not necessarily referred to, and only part of them may beused for reference. When only part is referred to, burdens on thearithmetic portion 203 can be reduced. Similarly, only part of the groupof equations which are stored in the storage portion 204 may be used inthe simulation. Although only one object 301 is displayed on the fieldin FIG. 6B, the plurality of objects 301 may be displayed. In that case,the number of data or parameters such as a physical quantity to bereferred to or the group of equations may be appropriately adjusteddepending on the number of the objects 301 displayed on the displayscreen 102.

FIG. 6C illustrates motion of the object 301 in the case where theelectronic device 100 is deformed by being further curved to protrudedownward. For example, the spherical object 301 which has beenpositioned as illustrated in FIG. 6B falls into the hollow formed by thecurving of the display screen 102. The object 301 which has fallen intothe hollow is enclosed on both sides by high walls of the display screen102 and thus cannot move from side to side; the object 301 moves at thebottom of the hollow and stops after awhile.

In the above manner, data on the three-dimensional shape of the displayscreen 102 is calculated using the sensors 156 and the processor 151,whereby the object 301 which moves according to the shape can bedisplayed. Accordingly, the user can feel a sense of reality as if theobject 301 displayed on the display screen 102 exists on the surface ofthe electronic device 100.

Note that although the deformation of the display screen 102 is acondition for starting move of the object 301 here, the condition is notlimited thereto. For example, when acceleration sensors are provided asthe input interface 167 in the electronic device 100, detection ofacceleration by the acceleration sensors may be the condition forstarting move of the object 301. The object 301 can be made to move in adirection in accordance with inclination detected by the accelerationsensors. In addition, when the microphone 166 is used as the inputinterface 167, the object 301 can be made to move in accordance with thevolume of input sound, for example. Alternatively, the object 301 may bemade to move in direction in accordance with a direction input with thekeyboard 170. Further, the object 301 may be reset to its initialposition by a press of the home button 103, which is provided on thehousing 101 of the electronic device 100.

(Procedure for Display Processing of Object)

Next, a procedure for display processing of an object in the electronicdevice 100 according to one embodiment of the present invention isdescribed with reference to FIG. 7, FIG, 8, FIG. 9, FIG. 10, FIG. 11,FIG. 12, and FIG. 13.

FIG. 7 is a flow chart which shows a procedure 500 for displayprocessing of an object in the electronic device 100 according to oneembodiment of the present invention. Here, as an example of theprocedure for display processing of an object, an application fordisplaying the object is described. An example of such an application isa background of a home screen of the electronic device 100, and thelike.

Accordingly, the procedure for display processing of an object which isdescribed below is executed by an application program. The program isstored in a computer-readable storage medium such as the auxiliarymemory 154 or the main memory 152.

Further, although the program corresponds to software here, such aprocessing means can also be electronic circuit or mechanical hardware.

First, an application for displaying an object in the electronic device100 according to one embodiment of the present invention is started(S001). Here, the application program or the like stored in theauxiliary memory 154 is loaded into the main memory 152. Note thatstarting of the application may be set to be performed at the same timeas startup of the operating system.

Then, initial conditions related to the object and the field are set(S002 and S003). The initial conditions related to the object and thefield may be set so that conditions at the time of close of theapplication in the previous use are transferred, or may be reset to thedefault settings in every use. Moreover, the user may be allowed toselect any of these settings.

The initial conditions of the object set the object ID 251 which is theobject data 250 stored in the storage portion 204, display positions,and the like. Further, the initial conditions of the field set the fieldID 261 which is the field data 260 stored in the storage portion 204,and the like. Specifically, the setting refers to reading of the objectID 251 and the object physical quantity 253 and the like associatedtherewith from the auxiliary memory 154, and storing of them in the mainmemory 152. Similarly, parameters for the field data 260 are also set.

An order in which the setting of the initial conditions related to theobject (S002) and the setting of the initial conditions related to thefield (S003) are performed is not limited to the above. The setting ofthe initial conditions related to the field may be performed first, orthe setting of the initial conditions related to the object (S002) andthe setting of the initial conditions related to the field (S003) may beperformed at the same time.

Then, with the plurality of sensors 156 which are provided in thevicinity of the display screen 102, positional data in the respectivepositions is obtained (S004). After that, on the basis of the positionaldata obtained with the plurality of sensors 156, the three-dimensionalshape of the display screen 102 is calculated in the arithmetic portion203 (S005). In the case where the shape of the display screen 102 hasalready been obtained with another system or application after startingof the electronic device 100 and before starting of the application, thethree-dimensional shape of the display screen 102 may be calculatedusing the obtained data on the shape.

Then, the shape of the display screen 102 calculated in Step S005 iscompared with the initial state of the object, and whether to make theobject move is determined (S006). For example, in the case where theinitial position of the object which is set in Step S002 corresponds toa position having an inclination that exceeds a predetermined thresholdwith respect to a horizontal plane due to a curved shape of the displayscreen 102, the arithmetic portion 203 determines that move of theobject is necessary. In contrast, in the case where the initial positionof the object corresponds to a position parallel to the horizontal planealthough the display screen 102 has a curved portion, the object is atrest, and thus the arithmetic portion 203 determines that move of theobject is not necessary. When the arithmetic portion 203 determines thatmove of the object is necessary, the processing shifts into an objectmoving mode 510 (or an object moving mode 520 which will be described asanother example) (S007). When the arithmetic portion 203 determines thatmove of the object is not necessary, the processing shifts into anobject standby mode 530 (S008).

When the object standby mode 530 is excited, whether to close theapplication is judged in Step S009; in the case where the conditions aremet (i.e., in the case where an application closing flag which will bedescribed later is “1”), the application is closed (S010), and in thecase where the conditions are not met (i.e., in the case where theapplication closing flag which will be described later is “0”), theprocessing returns to Step S004.

At the time of closing the application, various kinds of data stored inthe main memory 152 is preferably stored in the auxiliary memory 154.Accordingly, the data can be used as the initial settings in the nextstarting of the application.

FIG. 8 is a flow chart which shows the object moving mode 510 in aprocedure for display processing of an object in the electronic device100 according to one embodiment of the present invention. In the objectmoving mode 510, simulation of motion of the object is repeated and theresults are displayed on the display screen 102 each time. A repetitionof the simulation and display of the object is performed every givenperiod (hereinafter referred to as Δt).

After the object moving mode starts (S020), the motion of the object isdetermined according to the shape of the display screen which hasalready been calculated (S021). Here, the motion of the object isdetermined before Δt.

The motion of the object is simulated according to a predetermined lawin the arithmetic portion 203. The simulation is executed using thephysical quantity of the object, the physical quantity of the field, andthe like which are set in Step S002 and Step S003. Through thissimulation, the moving direction, speed, and the like of the object aredetermined. For the simulation of the moving direction, speed, and thelike of the object, the field data 260 on the associated field ID 261 isalso referred to. For example, in the case where the field physicalquantity 263 of the field ID 261 which is selected includes data on anelastic material such as rubber, the motion of the object which bouncesafter falling into a hollow formed by curving the display screen 102 issimulated. Data on the simulated motion of the object is stored in themain memory 152 of the storage portion 204.

Simulation for displaying rotation, deformation, or the like of theobject may be performed to further provide the user with a sense ofreality. However, when the simulation is complicated, the arithmeticportion 203 becomes overloaded so that delay in processing of thearithmetic portion 203 inhibits an appropriate display; as a result, asense of reality offered to the user is diminished. In view of this, itis preferable that computation be simplified or a database ofcomputation results be made, for example, so as to avoid a slowdown inthe computation speed.

Then, according to the motion of the object which has been determined bythe simulation, the motion of the object is displayed on the displayscreen 102 for a period of Δt (S022). The motion of the object isaffected by coefficient of friction, air resistance, gravity, or thelike depending on the field physical quantity 263. Note that motion ofthe object includes stop of the object. Therefore, Step S023 includes amotion sequence from stop of the object to move of the object, and thatfrom move of the object to stop of the object.

Then, the three-dimensional shape of the display screen 102 isrecalculated (Step S023). Specifically, in this step, data stored in thestorage portion 204, which is data on the previously calculatedthree-dimensional shape of the display screen 102, is rewritten withdata on the three-dimensional shape of the display screen 102 which isnewly obtained by a recalculation sequence 511 for the three-dimensionalshape of the display screen.

Here, the recalculation sequence 511 for the three-dimensional shape ofthe display screen is described with reference to FIG. 9. When therecalculation sequence 511 for the three-dimensional shape of thedisplay screen starts (S040), positional data of given parts of thedisplay screen 102 is obtained again by the sensors 156 (S041). Then,the three-dimensional shape of the display screen 102 is recalculated onthe basis of the positional data (S042). At the same time as therecalculation, data on the three-dimensional shape of the display screen102 is stored in the storage portion 204, and after that, therecalculation sequence 511 for the three-dimensional shape of thedisplay screen ends (S043).

By the recalculation sequence 511 for the three-dimensional shape of thedisplay screen, the data on the previously calculated three-dimensionalshape of the display screen 102 is updated by being overwritten with thenewly obtained data on the three-dimensional shape of the display screen102. Alternatively, without overwriting, an address different from thatof the storage portion 204 in which the data on the previouslycalculated three-dimensional shape of the display screen 102 is storedmay be specified to store the newly obtained data.

Further, at the same time as the recalculation sequence 511 for thethree-dimensional shape of the display screen, data input with an inputdevice such as a touch panel may be read from the input interface 167 inStep S023.

Then, for update of data on the motion of the object which is stored inthe main memory 152, the motion of the object is mutated again (S024).The simulation of the motion of the object may be performed withoutexception; alternatively, it is possible to perform the simulation ofthe motion of the object only in the case where there is a differencebetween the data on the previously calculated three-dimensional shape ofthe display screen 102 and the data on the three-dimensional shape ofthe display screen 102 recalculated in Step S023 when these pieces ofdata are compared with each other. In this case, the amount of thechange preferably has a predetermined threshold value for the comparisonbetween the pieces of data on the three-dimensional shape, in which casenoise or burdens due to unnecessary computation by the arithmeticportion 203 in a region unnoticeable to the user can be reduced. Thatis, it is preferable that it be determined that the three-dimensionalshape of the display screen 102 has been changed only when the amount ofthe change of the data on the recalculated three-dimensional shape ofthe display screen 102 from the data on the previously calculatedthree-dimensional shape of the display screen 102 exceeds thepredetermined threshold value, and the processing proceed to Step S024;when there is no change, the processing proceed to Step S025.

Next, in Step S025, whether the object is in motion or at rest isdetermined. In the object moving mode 510, simulation of the motion ofthe object is performed every period of Δt. Therefore, in some cases,the results of the simulation in Step S021 show that the object after Δtis in motion. Further, depending on the results of the recalculation ofthe three-dimensional shape of the display screen 102 in Step S023, andthe like, another motion of the object might occur by the simulation inStep S024. In this case, it is determined that the object is not at restbut in motion, so that the processing returns to Step S022.

In contrast, in the case where it is determined that the object is atrest, the object moving mode 510 is exited (S026).

As described above, a display is performed with the period of the motionof the object divided into periods each having a length of Δt, and thesimulation is repeated with the shape of the display screen changed,whereby the motion of the object can be modified and displayed eachtime, which can have the user feel a sense of reality.

FIG. 10 is a flow chart which shows a procedure for processing in theobject moving mode 520 which is different from the procedure forprocessing described with reference to FIG. 8.

In the processing in the above-described object moving mode 510, timefor displaying the motion of the object is divided and simulation forthe next display is performed after an intended display is finished;meanwhile, in processing in the object moving mode 520, the motion ofthe object is displayed for a given period and in this period,simulation for the next display is performed.

When the object moving mode 520 starts (S050), the motion of the objectis determined by simulation on the basis of data on thethree-dimensional shape of the display screen 102 which has already beencalculated (S051).

Then, the motion determined by the simulation is displayed for a givenperiod (here, Δt), and during the period Δt, the motion of the object inthe next period Δt is simulated. The display of the motion of the objectis executed in accordance with a display sequence 521 for the motion ofthe object, and the next motion of the object is executed in accordancewith a computation sequence 522 for the motion of the object (S052).

Here, the display sequence 521 for the motion of the object is describedwith reference to FIG. 11. After the sequence starts (S060), the time tis initialized to “0” (S061). Then, in Steps S062 to S065, the motion ofthe object is displayed. That is, displaying the motion of the objectfor each time t (S063), updating t by addition of t (S064), anddisplaying the motion of the object again, are repeated until the time tbecomes Δt (S065). Accordingly, the motion of the object continues to bedisplayed until the time t becomes Δt. When the time t becomes Δt, thedisplay sequence 521 for the motion of the object ends (S066).

Next, the computation sequence 522 for the motion of the object isdescribed with reference to FIG. 12. In a manner similar to that of thedisplay sequence 521 for the motion of the object, after the sequencestarts (S070), the time t is initialized (S071). Then, in Steps S072 toS077, the motion of the object corresponding to the three-dimensionalshape of the display screen 102 in the period Δt is simulated.

With the sensors 156, positional data of given parts of the displayscreen 102 is obtained (S073). Then, the three-dimensional shape of thedisplay screen 102 is calculated on the basis of the obtained positionaldata (S074). In accordance with data on the calculated three-dimensionalshape of the display screen 102, the motion of the object for the periodΔt is simulated and determined (S075). After that, updating t byaddition of t is performed (S076).

Steps S073 to S076 described above are repeated until t becomes Δt(S077). Here, the period Δt is the same as the period Δt which isdetermined in the display sequence 521 for the motion of the object. Bysynchronization between the sequences, the sequences are concurrentlyperformed during the same period (Δt). That is, in the display sequence521 for the motion of the object, while the motion of the object isdisplayed, the motion of the object for the next period can bedetermined in advance by the computation sequence 522 for the motion ofthe object.

In the computation sequence 522 for the motion of the object, when theshape of the display screen 102 is changed during the period Δt, thedetermined motion of the object is modified by being overwritten. Notethat in FIG. 12, the simulation of the motion of the object is repeatedregardless of whether or not the shape of the display screen 102 ischanged; however, a step of judging change in the shape of the displayscreen 102 may be inserted, in which case the simulation may be skippedwhen the shape is not changed. Further, the calculation of thethree-dimensional shape of the display screen 102 may be performed onlywhen the positional data obtained by the sensors shows a change greaterthan or equal to a predetermined value.

When t becomes Δt as a result of the operation repeated as describedabove, the computation sequence 522 for the motion of the object ends(S078).

After Step S052, as illustrated in FIG. 10, whether the object is inmotion or at rest is determined (S053). In the case where the object isin motion, the processing returns to Step S052, whereas in the casewhere the object is at rest, the object moving mode 520 is exited(S054), and the processing shifts into the object standby mode 530.

As described above, in the object moving mode 520 illustrated in FIG.10, a display of the motion of the object and simulation of the motionof the object for the next period are performed in parallel during agiven period. Such processing allows the user to feel a sense ofreality.

Next, the object standby mode 530 is described. FIG. 13 is a flow chartwhich shows a procedure for processing in the object standby mode 530.The object standby mode refers to internal processing in the electronicdevice 100 which is performed in a period in which the object displayedon the display screen 102 is at rest, in a state where the applicationis running.

When the object standby mode starts (S030), in Step S031, theapplication closing flag which shows whether or not the application isclosed and a mode exiting flag which shows whether or not the objectstandby mode 530 is exited are initialized. For example, both of theflags are, set to “0”.

Then, whether or not the display screen 102 is deformed is checked withthe use of the sensors 156 (S032) by comparing the data on thethree-dimensional shape of the display screen 102 which has already beencalculated in Step S005, Step S042, or Step S074 with the data on thethree-dimensional shape of the display screen 102 which is recalculatedin Step S032 (S032). Note that the amount of the change preferably has apredetermined threshold value for the comparison between the pieces ofdata on the three-dimensional shape, in which case noise or burdens dueto unnecessary computation by the arithmetic portion 203 in a regionunnoticeable to the user can be reduced. That is, it is preferable thatit be determined that the three-dimensional shape of the display screen102 has been changed only when the amount of the change of the data onthe recalculated three-dimensional shape of the display screen 102 fromthe data on the previously calculated three-dimensional shape of thedisplay screen 102 exceeds the predetermined threshold value, and theprocessing proceed to Step S034.

In the case where it is determined that the three-dimensional shape ofthe display screen 102 has been changed, the mode exiting flag is set to“1” (S034). In the case where there is no change in the shape of thedisplay screen 102, whether or not there is input from the inputinterface 167 is then checked. The input from the input interface 167here is limited to input that affects the motion of the object.Similarly, when there is input, the mode exiting flag is set to “1”(S034). In the case where there is neither change in the shape of thedisplay screen 102 nor the input from the input interface 167, theprocessing proceeds to the next step while the mode exiting flag remains“0”.

In Step S035, whether or not there is notification of an instruction forclosing the application is checked. When there is the notification, theapplication closing flag for the application is set to “1” (S036). Theinstruction for closing the application includes, for example, aninstruction for closing the application from the user through theprocessor 151 and an instruction for closing the application fromanother application, a program, or an operating system. In the casewhere there is not the notification of the instruction for closing theapplication, the application closing flag remains “0”, which is theinitial value.

The flag is judged in Step S037. That is, in the ease where one or bothof the mode exiting flag and the application closing flag are set to“1”, the object standby mode is exited (S038). In contrast, in the casewhere both of the flags are “0”, the object standby mode persists. Inother words, the processing returns to Step S032.

As described above, in the object standby mode 530, an input state ismonitored by looping through checking of whether or not there is giveninput. When there is either kind of input, the object standby mode 530is exited and the processing proceeds to Step S009.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

Embodiment 2

Although the object 301 which is a spherical solid is described inEmbodiment 1 as an example of an object, an object is not limitedthereto. In this embodiment, the case where the object is designed toresemble liquid is described with reference to FIGS. 14A to 14C.

FIG. 14A is a perspective view illustrating the electronic device 100 inwhich a liquid object 303 is displayed on the display screen 102. In asimilar manner to the case of the electronic device 100 illustrated inFIGS. 6A to 6C, in order to describe display operation for the object303, other displays are not shown in the drawing; however, any otherdisplay matters such as a background image, an icon, a toolbar, apointer, a window, a text, a moving image, or a web browser can beactually displayed at the same time as the object 303.

The object 303 in FIG. 14A is designed to resemble a liquid body and isat rest as a given position. The range of motion of the object 303 isdefined as a field, which functions as a ground that affects the motionof the object 303.

FIG. 14B illustrates the electronic device 100 whose right end portionis curved by being lifted. By the deformation of the electronic device100, the display screen 102 is also deformed. The motion of the liquidobject 303 is simulated by the arithmetic portion 203 so that the objectmoves according to the shape of the display screen 102. In FIG. 14B,since the right side of the display screen 102 is lifted, the liquidobject 303 moves to the center of the display screen 102 as if it isattracted by gravity (in a direction represented by an arrow in thedrawing).

The object 303 is defined as liquid by the object physical quantity 253in the object data 250. The arithmetic portion 203 performs simulationby referring to data or a parameter stored in the object physicalquantity 253 and indicating that the object is liquid. Simulation inwhich the object 303 moves like real liquid according to a law of fluidmechanics (hydraulics) is performed, whereby the motion of the object isdisplayed as if liquid flows from higher to lower areas. As the data orparameter indicating that the object is liquid, for example, a value ofdensity, viscosity, a compaction property, surface tension, or the likemay be prepared as a physical quantity, or for simplicity, the degree ofa physical quantity such as viscosity may be divided into several levelsto be prepared as several pieces of data. Further, an image set ofliquid in various forms is preferably stored in the object image 254 inadvance so to provide the user with visual recognition of the liquidflowing.

A physical quantity and the like are set also for the field; forexample, the liquid object moves while encountering friction determinedfor the field.

Applying various laws to the motion of the liquid object 303 asdescribed above allows the user to feel a sense of reality.

Note that all of the physical quantities and the like of the object 303and the field are not necessarily referred to, and only part of them maybe used for reference. When only part is referred to, burdens on thearithmetic portion 203 can be reduced. Similarly, only part of the groupof equations which are stored in the storage portion 204 may be used inthe simulation. Although only one object 303 is displayed on the fieldin FIG. 14B, the plurality of objects 303 may be displayed. For example,in the case where more complicated computation is possible, by beingmade to move, the liquid object 303 can be divided into a plurality ofobjects or the plurality of objects can be combined into one object. Inthis case, each of the objects may be provided with the object ID 251 inresponse to generation and disappearance of the objects so as to bedefined, or the state of one object having one object ID which isdivided into a plurality of objects may be stored as data of the objectshape 252 or the like. Further, for simplification of the computation,an image expressing splash may be displayed around the liquid object 303appropriately.

FIG. 14C illustrates motion of the liquid object 303 in the case wherethe electronic device 100 is deformed by being further curved toprotrude downward. For example, the liquid object 303 which has beenpositioned as illustrated in FIG. 14B flows into the hollow formed bythe curving of the display screen 102. The object 303 which has flowedinto the hollow is enclosed on both sides by high walls of the displayscreen 102 and thus cannot move from side to side; the object 303collects at the bottom of the hollow and stops after a while.

In the above manner, data the three-dimensional shape of the displayscreen 102 is calculated using the sensors 156 and the processor 151,whereby the liquid object 303 which moves according to the shape can bedisplayed. Accordingly, the user can feel a sense of reality as if theliquid object 303 displayed on the display screen 102 exists on thesurface of the electronic device 100.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

Embodiment 3

In this embodiment, an electronic device in which a touch panel is usedas an input device included in an input portion and the object can becontrolled by input from the touch panel is described with reference toFIGS. 15A and 15B.

FIG. 15A is a perspective view illustrating the electronic device 100which is curved. An object 302 which is a spherical solid is displayedon the right end portion on the display screen 102 of the electronicdevice 100. When a user touches the object 302 (actually the usertouches the touch panel on the display screen 102 where the object 302is displayed) and flick the display screen 102 to the left (in adirection denoted by an arrow in FIG. 15A) with his/her finger, forexample, the object 302 rolls in the direction of the flick, in responseto the motion of the finger. In FIG. 15B, the display screen 102 iscurved at the center to have a steep step from right to left; thus, theobject 302 is displayed as if the object 302 falls down and rolls due tothis curve.

With such a structure in which the electronic device 100 includes thetouch panel in the input portion, the procedure for display processingof the object is the same as that described in Embodiment 1 inprinciple. However, the conditions for starting the motion of the objectare access to the object through the touch panel as well as detection ofdeformation of the display screen 102, which is described inEmbodiment 1. Therefore, in the procedure for processing described inEmbodiment 1, the steps for determining move of an object (S006, S025,and S053) involve checking of whether or not there is detection of inputwith the touch panel.

In a similar manner, in the object moving mode 510 illustrated in FIG.8, data on input from the touch panel is updated in the step forupdating concerning change in the three-dimensional shape of the displayscreen (S023). Further, in the object moving mode 520 illustrated inFIG. 10, data on input from the touch panel is checked in the step forobtaining positional data with the sensors (S073) and the step forcalculating the three-dimensional shape of the display screen (S074).Further, in the object standby mode 530 illustrated in FIG. 13, data oninput from the touch panel is checked in the step for checking whetheror not there is input from the input interface 167 (S033).

Note that although the example in which the touch panel is used formoving the object 302 is described in this embodiment, the object 302may be made to move using, other than the touch panel, any of a varietyof input devices which can be included in the input portion 205; forexample, a housing switch provided on the housing 101, a keyboard whichis incorporated in or externally provided to the electronic device 100,a pointing device (e.g., a mouse or a controller) which is connected tothe external port, a microphone for sound input, and the like can beused. When a microphone is used, for example, the motion of the object302 can be appropriately adjusted to be displayed depending on thevolume of input sound (see FIG. 16A). It is also possible to analyzeinput of the user's voice “move leftward”, recognize the meaning, andmake the object move leftward in accordance with the content, forexample (see FIG. 16B). Further, a gesture device which detects gesture(motion) of a user and converts it into an input signal may be providedin the input portion 205 of the electronic device 100 to move the object302.

As described above, the input portion 205 can include a variety of inputdevices with which the object 302 displayed on the display screen 102can be made to move, so that the user can feel a sense of reality as ifthe object exists on the surface of the electronic device 100.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

Embodiment 4

In this embodiment, the case where processing for displaying an objecton the display screen is executed at the same time as a differentapplication is described with reference to FIGS. 17A and 17B.

In FIG. 17A, different application software is running and an electronicdocument is displayed in a window 305 of the application software on thedisplay screen 102. Further, on the display screen 102, a plurality oficons 304 of the home screen which is displayed by the operating systemof the electronic device 100 are displayed.

Although text-editing application software is illustrated as thedifferent application software in FIG. 17A, the present invention is notlimited thereto; various kinds of application software for spreadsheets,database administration, phone calls, e-mailing, web browsers, blogs,videoconference, music reproduction, moving image reproduction, digitalcameras, digital video cameras, e-book readers, computer games, and thelike are included.

An object 306 a and an object 306 b according to one embodiment of thepresent invention are displayed on the display screen 102. The object306 a and the object 306 b move in response to input from the sensors orthe input portion described in Embodiments 1 to 3. Here, the object 306a and the object 306 b are set so that the objects are not allowed tomove in the entire region of the display screen 102 and cannot entergiven regions. That is, in FIG. 17A, the regions in which the pluralityof icons 304 and the window 305 of the application software aredisplayed are set as the regions where the objects cannot enter, so thatthe object 306 a and the object 306 b move while avoiding the regions.In addition, when these regions where the objects cannot enter areregarded as bodies with given physical quantities, the motion of theobjects is set to be defined depending on the physical quantities.

Further, as illustrated in FIG. 17B, motion of an object 306, whichmoves in accordance with deformation of the display screen 102, is alsoaffected by the icons 304. Thus, the actual shape of the display screen102 and the shapes of the virtual icons 304 similarly influence themotion of the object 306, so that the user can further feel a sense ofreality.

In this manner, when not only an object and a field but also a varietyof images displayed on the display screen 102, such as icons or windowsof application software have physical quantities to be treated asbodies, the motion of the object can be more realistic.

Note that a target which is provided with a physical quantity andtreated as a body is not limited to an image displayed on the displayscreen and may be ends (four sides) of the display screen, a finger of auser which is detected with the touch panel, or the like.

In order to achieve such a display, images such as icons or windows canbe regarded as second objects and object IDs, object shapes, objectphysical quantities, or the like can be set in advance. The conditionsof the second objects may be incorporated in simualation of the motionof the object by the arithmetic portion.

Alternatively, images such as icons or windows may be defined as partsof a field and integrated into field data to be reflected in thesimulation of the motion of the object.

Note that not all the images such as the icons or windows arenecessarily regarded as bodies; only predetermined icons or windows maybe regarded as bodies. Alternatively, the object may be displayed movingon these images so that the images are regarded as a background for theobject without being regarded as bodies. It is also possible to allowthe user to decide whether to regard these images as bodies.

In the case of performing such processing of the motion of the objectwhich incorporates images such as icons or windows of applicationsoftware or ends of the display screen as bodies, a display of theseimages and the like is controlled by the operating system; thus, theprocessing needs to interlock with the operating system. Therefore, byincorporating the procedure for display processing of the object intothe operating system as a module, a series of steps of displayprocessing of the object can be executed efficiently.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

Embodiment 5

In this embodiment, display processing in which a given region of acurved display screen is selected and an object moves only in the regionis described with reference to FIGS. 18A and 18B.

FIGS. 18A and 18B are perspective views of the electronic device 100 inwhich an object 307 is displayed on part of the curved display screen102. The display screen 102 is curved at the center to have a steep stepfrom right to left.

In FIG. 18A, the object 307 is displayed on a high right part (a region308 a) of the curved display screen 102. The object 307 recognizes thehigh part of the curved display screen 102, i.e., the region 308 a andmoves selectively in the region 308 a.

In FIG. 18B, the object 307 is displayed on a low left part (a region308 b) of the curved display screen 102. The object 307 recognizes thelow part of the curved display screen 102, i.e., the region 308 b andmoves selectively in the region 308 b.

Such a display of an object can be used for a home screen or ascreensaver of the operating system or various kinds of applicationsoftware.

To perform the above display of an object, the procedure for processingwhich is described in Embodiment 1 can be used. However, in the objectmoving mode of the procedure for processing, a region in which an objectcan move needs to be defined in accordance with the three-dimensionalshape of the display screen 102 as appropriate. Thus, in the objectmoving mode, the procedure for processing which is illustrated in FIG.19 is employed, for example.

FIG. 19 is a flow chart which illustrates a procedure for processing inan object moving mode 540 in this embodiment, which is obtained byadding some steps to the object moving mode 510. After the object movingmode starts (S080), a region in which the object can move is determinedin Step S081 on the basis of data on the three-dimensional shape of thedisplay screen 102 which has already been calculated by the arithmeticportion 203.

In this embodiment, the region in which the object can move is definedas a given height of the display screen 102. For example, when themiddle position of the display screen is set as a position of reference,a two-dimensional space positioned above (or below) the position ofreference by a given height can be defined as the region in which theobject can move. Alternatively, a three-dimensional space positionedabove (or below) the position of reference by a given range of heightscan be defined as the region in which the object can move. Such atwo-dimensional or three-dimensional space may be defined as a value inadvance, or a given space including a portion in which an object isdisplayed may be defined as a region in which the object can move.

After the region which the object can move is determined, motion of theobject in the region is determined in Step S082. In simulation of themotion of the object at this time, data on the region which isdetermined in Step S081 is used as boundary conditions. Subsequent stepsof processing for a display and the like (S083 to S089) are similar tothose in the object moving mode 510 which is illustrated in FIG. 8. Notethat in the case where the shape of the display screen 102 is changedduring motion of the object, a region in which the object can move isdetermined again in Step S086, and a display of the motion of the objectis overwritten.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

Embodiment 6

In this embodiment, a computer game which utilizes a display of anobject moving in accordance with the shape of the display screen isdescribed with reference to FIGS. 20A and 20B.

FIGS. 20A and 20B are perspective views illustrating the electronicdevice 100 in which images of a computer game are displayed on thedisplay screen 102. In FIG. 20A, the display screen 102 is planar. Ariver 309 runs through the center of the display screen 102, and acharacter 310 a and a character 310 b which are controllable by the userare displayed on the left bank of the river 309. In this computer game,the river 309 running through the center prevents the character 310 aand the character 310 b from moving to the right bank.

Under the circumstances, as illustrated in FIG. 20B, the user curves thedisplay screen 102 such that the center of the display screen 102protrudes downward (the user moves the display screen 102 in thedirections denoted by arrows in FIG. 20B), whereby the river 309 movesto the part bended downward and both of the banks are brought close toeach other; thus, the character 310 a and the character 310 b can bemade to move to the right bank.

Here, by setting the river 309 as the above-described object, water flowin the river 309 interlocks with deformation of the display screen 102,whereby the user can feel a sense of reality. In addition, bycalculation of the shape of the display screen 102 using the sensors156, the range of motion of the characters can be controlled inaccordance with the shape.

To achieve such a computer game, the procedures for display processingof objects which are described in any of Embodiments 1 to 5 can beapplied to bodies including characters in the computer game. Theprocedure for display processing of an object may be incorporated in acomputer program as a module, or may be executed while interlocking withan operating system in which the procedure for display processing of anobject is incorporated. Such a program, a module, or the like may bestored in a computer-readable storage medium such as the auxiliarymemory 154 or the main memory 152, or alternatively such a processingmeans may be achieved in the form of an electronic circuit or mechanicalhardware.

As described above, the game can proceed depending on change in theshape of the display screen 102, so that the user can enjoy reality ofthe computer game.

This embodiment can be combined with any of the other embodiments asappropriate. Note that the present invention includes, in its scope, amethod, hardware (e.g., an electronic device, a computer, asemiconductor device, or a storage medium), a system, a program,software, and the like in which the operations in the present inventionare achieved.

REFERENCE NUMERALS

100: electronic device, 101: housing, 102: display screen, 103: homebutton, 104: volume control button, 105: mute button, 106: microphone,107: speaker, 108: sleep button, 109: camera, 150: system bus, 151:processor, 152: main memory, 153: memory controller, 154: auxiliarymemory, 155: sensor controller, 156: sensor, 157: display controller,158: display device, 159: power supply controller, 160: power supply,161: communication controller, 162: communication I/F, 163: soundcontroller, 164: speaker, 165: sound output connector, 166: microphone,167: input interface, 168: housing switch, 169: touch panel, 170:keyboard, 171: camera, 172: external port, 173: output interface, 174:vibration motor, 180: RAM, 181: ROM, 182: operating system, 183:application program, 184: program module, 185: program data, 186: BIOS,187: display portion physical property data, 188: sensor characteristicsdata, 201: display portion, 202: detection portion, 203: arithmeticportion, 204: storage portion, 205: input portion, 206: output portion,250: object data, 251: object ID, 252: object shape, 253: objectphysical quantity, 254: object image, 255: default of an object, 260:field data, 261: field ID, 262: field shape, 263: field physicalquantity, 264: field image, 265: default of a field, 301: object, 302:object, 303: object, 304: icon, 305: window, 306: object, 306 a: object,306 b: object, 307: object, 308 a: region, 308 b: region, 309: river,310 a: character, and 310 b: character.

This application is based on Japanese Patent Application serial no.2012-109132 filed with Japan Patent Office on May 11, 2012, the entirecontents of which are hereby incorporated by reference.

1. (canceled)
 2. A displaying method of an electronic device comprising the steps of: displaying an object on a display screen of the electronic device, the display screen having flexibility; obtaining positional data of the object on the display screen and a field on which the object is configured to move; calculating a three-dimensional shape of the display screen based on the positional data; checking whether or not there is touch input on the object; and determining move of the object on the field based on the touch input and the calculation result.
 3. The displaying method according to claim 2, further comprising a step of comparing the calculated three-dimensional shape with an initial state.
 4. The displaying method according to claim 2, wherein the display screen is curved to have a steep step and the object is displayed to fall down and roll due to the steep step.
 5. The displaying method according to claim 2, further comprising a step of simulating the move of the object.
 6. The displaying method according to claim 2, wherein the electronic device comprises a transistor comprising an oxide semiconductor.
 7. The displaying method according to claim 2, further comprising a step of generating sound to alert when a degree of curve of the display screen exceeds a predetermined threshold value.
 8. The displaying method according to claim 2, further comprising a step of analyzing voice with a microphone such that the move of the object is controlled in response to the voice.
 9. The displaying method according to claim 2, further comprising a step of recalculating the three-dimensional shape of the display screen.
 10. The displaying method according to claim 2, wherein the electronic device is a portable electronic device selected from a mobile phone, a personal computer, a personal digital assistant, an electronic paper, or a portable game machine.
 11. A displaying method of an electronic device comprising the steps of: displaying an object on a display screen of the electronic device, the display screen having flexibility; obtaining positional data of the object on the display screen and a field on which the object is configured to move; calculating a three-dimensional shape of the display screen based on the positional data; comparing the calculated three-dimensional shape with an initial state; checking whether or not there is touch input on the object; and determining move of the object on the field based on the touch input and deformation of the display screen detected by the comparison.
 12. The displaying method according to claim 11, wherein the display screen is curved to have a steep step and the object is displayed to fall down and roll due to the steep step.
 13. The displaying method according to claim 11, further comprising a step of simulating the move of the object.
 14. The displaying method according to claim 11, wherein the electronic device comprises a transistor comprising an oxide semiconductor.
 15. The displaying method according to claim 11, further comprising a step of generating sound to alert when a degree of curve of the display screen exceeds a predetermined threshold value.
 16. The displaying method according to claim 11, further comprising a step of analyzing voice with a microphone such that the move of the object is controlled in response to the voice.
 17. The displaying method according to claim 11, further comprising a step of recalculating the three-dimensional shape of the display screen.
 18. The displaying method according to claim 11, wherein the electronic device is a portable electronic device selected from a mobile phone, a personal computer, a personal digital assistant, an electronic paper, or a portable game machine.
 19. A displaying method of an electronic device, the electronic device comprising a display portion, a detection portion, and an arithmetic portion, the displaying method comprising the steps of: displaying an object on a display screen in the display portion, the display screen having flexibility; obtaining positional data of the object on the display screen and a field on which the object is configured to move using a plurality of sensors in the detection portion; calculating a three-dimensional shape of the display screen based on the positional data using the arithmetic portion; checking whether or not there is touch input on the object; and determining move of the object on the field based on the touch input and the calculation result.
 20. The displaying method according to claim 19, further comprising a step of comparing the calculated three-dimensional shape with an initial state.
 21. The displaying method according to claim 19, wherein the display screen is curved to have a steep step and the object is displayed to fall down and roll due to the steep step.
 22. The displaying method according to claim 19, further comprising a step of simulating the move of the object.
 23. The displaying method according to claim 19, wherein the electronic device comprises a transistor comprising an oxide semiconductor.
 24. The displaying method according to claim 19, further comprising a step of generating sound to alert when a degree of curve of the display screen exceeds a predetermined threshold value.
 25. The displaying method according to claim 19, further comprising a step of analyzing voice with a microphone such that the move of the object is controlled in response to the voice.
 26. The displaying method according to claim 19, further comprising a step of recalculating the three-dimensional shape of the display screen.
 27. The displaying method according to claim 19, wherein the electronic device is a portable electronic device selected from a mobile phone, a personal computer, a personal digital assistant, an electronic paper, or a portable game machine. 